TWISTED CHARACTER OF A SMALL REPRESENTATION OF GL(4) 



YuvAL Z. Flicker and Dmitrii Zinoviev 

Abstract. We compute by a purely local method the (elliptic) 0-twisted character Xt^y of 
the representation tty = I{3,i)i^3 x Xy) of G = GL(4, F), where is a p-adic field, p 7^ 2, and 
Y is an unramified quadratic extension of F; xy is the nontrivial character of F^ /Ny/pY'^ . 

The representation is normalizedly induced from ^ "0^ ) ^ ™i £ GL(i, F"), 

on the maximal parabolic subgroup of type (3, 1); 9 is the "transpose-inverse" involution of G. 

We show that the twisted character Xny of ''^Y is an unstable function: its value at a twisted 
regular elliptic conjugacy class with norm in Cy = Cy(i^)="(GL(2,y)/i<"^)j?" is minus its 
value at the other class within the twisted stable conjugacy class. It is at the classes without 
norm in Gy . Moreover Try is the endoscopic lift of the trivial representation of Gy . 

We deal only with unramified Y/ F ^ as globally this case occurs almost everywhere. The 
case of ramified Y/F would require another paper. 

OurCr ="(Rr/FGL(2)/GL(l))F" has Y-points CrC^^) = {{g,g') 6 GL(2, y) x GL(2, F); 
det(g) = de\.{g')}/Y^ {Y^ embeds diagonally); cr(/ 1) in Gal(Y/i^) acts by a{g,g') = 
{ag',ag). It is a 0-twisted elliptic endoscopic group of GL(4). 

Naturally this computation plays a role in the theory of lifting of Gy and GSp(2) to GL(4) 
using the trace formula, to be discussed elsewhere. 

Our work extends - to the context of nontrivial central characters - the work of [FZ4] , where 
representations of PGL(4, F) are studied. In [FZ4] a 4-dimensional analogue of the model of 
the small representation of PGL(3, F) introduced with Kazhdan in [FK] in a 3-dimensional 
case is developed, and the local method of computation introduced in [FZ3] is extended. As 
in [FZ4] we use here the classification of twisted (stable) regular conjugacy classes in GL(4, F) 
of [F], motivated by Weissauer [W]. 



Introduction 

Let TT be an admissible representation (see Bernstein- Zelevinsky [BZ], 2.1) of a p-adic 
reductive group G. Its character Xn is a complex valued function defined by tr ^{fdg) = 
Jg X-K{9)f{g)d9 for all complex valued smooth compactly supported measures fdg ([BZ], 
2.17). It is smooth on the regular set of the group G. The character is important since it 
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characterizes the representation up to equivalence. A fundamental result of Harish-Chandra 
[H] establishes that the character is a locally integrable function in characteristic zero. 

Let 6 be an automorphism of finite order of the group G. Define ^tt by ^'nig) — TT{9{g)). 
When TT is invariant under the action of 6 (thus ^tt is equivalent to tt), Shintani and others 
introduced an extension of tt to the semidirect product G xi (9). The twisted character 
Xnig X 9) is defined by in:{fdg x 9) = jQXnig x 9)f{g)dg for all fdg. It depends only 
on the ^-conjugacy class {hg9{h)~^; h E G} oi g. It is again smooth on the ^-regular set, 
and characterizes the ^-invariant irreducible tt up to isomorphism. Moreover, it is locally 
integrable (see Clozel [C]) in characteristic zero. 

Characters provide a very precise tool to express a relation of representations of different 
groups, called lifting, initiated by Shintani and studied extensively in the case of base 
change, and also in non base change situations such as twisting by characters (Kazhdan [K] , 
Waldspurger [Wa]), and the symmetric square lifting from SL(2) to PGL(3) ([Fsym], [FK]). 
In this last case twisted characters of ^-invariant representations of PGL(3) are related to 
packets of representations of SL(2), and 9 is the involution sending g to its transpose-inverse. 

The aim of the present work is to compute the twisted {by 9) character of a specific 
representation tt = tty = -f(3,i)(l3 x Xy), of the group G = GL(4, F), F a p-adic field, p 
odd. Here Y/F is an unramified quadratic extension and xy is the quadratic character of 
F^ which is trivial on the group Ny/F^^ ■, where Nyjp is the norm map from Y to F . This 
TT is normalizedly induced from the representation ^ XYij^i)-, ^ GL(z,F), of 

the standard (upper triangular) maximal parabolic subgroup P of type (3, 1). It is invariant 
under the involution 9{g) — J~^*g~^J, where J = {aiSi^^-j), ai = 02 = 1, as = 04 = —1. 

A natural setting for the statement of our result is the theory of liftings to the group 
G = GL(4) from its ^-twisted endoscopic (see Kottwitz-Shelstad [KS]) F-group 

Cy = {ig,g') e GL(2) x GL(2); det^ = det^'}/G^, 

where the multiplicative group — GL(1) embeds as 2; 1— > {zl2,zl2), I2 is the identity 
2x2 matrix, with Gal(F/F)-action which is a composition of the usual Galois action on 
each of the two factors GL(2) with the transposition {g,g') ^ {g'lg) if c £ Gal(F/F) has 
nontrivial restriction to Y . Here F is a separable algebraic closure of F containing Y . 

The corresponding map \y of dual groups is simply the natural embedding in G = 
GL(4, C) of the non connected Cy = Z^{s9) 

= {geG = GL(4, C); gsJ'g = ^ J = ( ) } = » ( ( ^ ) ' 

= ((:f:B)'^=(jjJ;(^=(::)'^)MGL(2,C)^;detA.det5 = l}/CX^, 

where z E embeds as the central element (2,2"^), and where s = diag(— 1, 1, — 1, 1) 
and CO = (^^^~q^- Thus Cy is the ^-centralizer in G of the semisimple element s (and 9 is 

defined on G by the same formula that defines 6* on G), and Gal(F/F) acts via conjugation 
by L. 
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Indeed, our result can be viewed as asserting that the ^-invariant representation tt of 
G = G(F), whose central character is 7^ 1 of order two, is the endoscopic lift of the 
trivial representation of Cy — Cy(F) = {GL{2,Y)/F^)f. The subscript F here indicates 
that (GL(2,y)/F^)F consists of g in GL(2,y)/F^ with det{g) in 

To state this we note that the embedding Ay : Cy G defines a norm map. This norm 
map relates the stable ^-conjugacy classes in G with stable conjugacy classes in Cy, where 
"stable" means the elements in G of an orbit in the points of G in a separable algebraic 
closure F of F. The crucial case is that of 6'-elliptic elements. A stable ^-conjugacy class 
consists of several ^-conjugacy classes. The stable 0-conjugacy classes of elements in G, and 
the 6'-conjugacy classes within the stable 6'-classes, have been described recently in [F], in 
analogy with the description of the (stable) conjugacy classes in the group of symplectic 
simihtudes GSp(2,F) of Weissauer [W]. In fact in [FZ4] the simpler case of PGL(4, F) is 
used, but here, as in [F], we deal with ^-classes in GL(4, F). We give here full details of the 
description in our case. 

There are four types of ^-elliptic elements of G, named in [F] and here I, II, III, IV, 
depending on their splitting behaviour. As in [F] , our work relies on an explicit presentation 
of representatives of the ^-conjugacy classes within the stable such classes in G. We present 
here the same set of representatives as in [FZ4] . 

The norm map, which we describe explicitly here, relates ^-conjugacy classes of types II 
and IV to conjugacy classes in Cy- It does not relate classes of types I, III to classes in 
Cy. Our "quadratic" case behaves then in a complementary fashion to that of [FZ4], where 
^-conjugacy classes of types I, III are related to conjugacy classes in the group C = SO (4) 
of [FZ4], but 6'-conjugacy classes of types II, IV arc not related to conjugacy classes in C. 

The stable 0-conjugacy classes of types II and IV come associated with a quadratic 
extension E/Es, where Y — E^ is a quadratic extension of F. The two 6'-conjugacy classes 
gr within the stable ^-classes are parametrized by r in E^ /Ne/EzE^ ■ We prove 

Theorem. The value of the -character Xtt{9 x d) of n = ny at the 6 -regular element 
g = gr of type II or IV, mmltiplied by a suitable Jacobian -^^^Wg)' ^"^ 2K(r); here k is the 
^1 character of E^ /Ne/E:,E'^ ■ At any 6 -regular element g of type I or III, Xnid x 9) = 0. 

In particular the character x-k{9 x ^) is an unstable function, namely its value at one 
6'-conjugacy class within a stable 6'-conjugacy class of type II or IV is negative its value at 
the other ^-conjugacy class. 

We deal only with unramified Y/F, as globally this case occurs almost everywhere. The 
case of ramified Y/ F would require another paper. 

Our result is a special case of the lifting with respect to Xy to the group G = GL(4, F) 
of representations of the group Cy — (GL(2, Y)/ F^)f- 

Our work develops the method of [FZ4] to the context of representations with nontrivial 
central characters. We use a model of our representation -Ky = /(3,i)(l3 x Xy)i different 
from the standard model of a parabolically induced representation. It is a twist of the four 
dimensional analogue of [FZ4], dealing with 7r4 = /(3 i)(l3 x 1), of a (three dimensional) 
model introduced and used with Kazhdan in [FK] to compute the twisted (by transpose- 
inverse) character of the representation tts = /(2,i)(l2) of PGL(3, F) normalizedly induced 
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from the trivial representation of the maximal parabolic subgroup. We do not use our 
results to prove the fundamental lemma since in our case, as well as that of [FZ4], the 
fundamental lemma is already established in [F] . In the case of the symmetric square lifting 
from SL(2, F) to PGL(3, F), an analogous purely local and simple proof of the fundamental 
lemma was given in [Fsym. Unit elements]. 

The work of [FK] uses local arguments to compute the twisted character of tts on one 
of the two twisted conjugacy classes within the stable one (where the quadratic form is 
anisotropic), and global arguments to reduce the computation on the other class (where the 
quadratic form is isotropic) to that computed by local means. A purely local computation 
for the second class is given in [FZ3]. In [FZ4] this local computation is developed in a four 
dimensional projective case. A global type of argument as in [FK] is harder to apply as there 
are not enough anisotropic quadratic forms in the four dimensional case. Anyway, a simpler, 
local proof, is better. Here we extend the work of [FZ4] to ^-invariant representations of 
GL(4, F) whose central character is nontrivial, necessarily quadratic. Our work is parallel 
to - but entirely independent of - the work of [FZ4]. 

Conjugacy classes 

Let F be a local nonarchimedean field, and R its ring of integers. Put G = GL(4), 
G = G{F) and K = G{R). Put Cy = {{91,92) e GL(2) x GL(2); det{gi) = dct(£/2)}/Gm 
{Grn embeds diagonally), viewed as a group over F with Galois action T{g,g') = {rg^rg') 
unless T G Gal{F/F) has nontrivial restriction to Y, in which case T{g, 9') — (rg', rg), where 
T{9ij) — (rgij). It is a form of the group C of [FZ4], and in particular Cy(F) = C(F), 
but the Gal(F/F)-action is different: r e Gal(F/F) takes {9,9') of C(F) to {rg^rg'). 
Then Cy = Cy{F) = {9 e GL{2,Y)/F''; detig) e F><} and Kc^ = Cy{R). Set 9{6) = 
j-it^-ij fQj. ^ Q jjgpe J Q ^, where w = (^^ g)' ^ separable algebraic 

closure F of F. The elements 5, 5' of G are called (stably) 6- conjugate if there is in G 
(resp. GL(4,F)) with 5' = g-^Se{g). 

Results of [F] concerning (stable) ^-twisted regular conjugacy classes are recalled in [FZ4], 
pp. 337-338. There are four types of 6'-elliptic classes, but the norm map N from G to Cy 
relates only the twisted classes in G of type II and IV to conjugacy classes in Cy. We 
should then expect the twisted character of the representation considered here to vanish on 
the twisted classes of type I and III. 

Norm map 

The norm map A'" : G — > Cy is defined on the diagonal torus T* of G by 

A"(diag(a, b, c, d)) = (diag(a6, cd), diag(ac, bd)). 

Since both components have determinant abed, the image of N is indeed in Cy. 

In type II we have a E , Ei = = F{-s/D), b E E^ , E2 = E"^ = F(VZD), and the 
norm map becomes 

A^(diag(a, 6, t6, era)) = (diag(a6, r(6)a(a)), diag(ar(5), 6(T(a))). 
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The two components on the right are mapped to each other by r, while the pairs of eigen- 
values {{ab,T{b)a{a)} and {ar(6), b(7{a)}) are permuted by a. Hence the right side defines 
a conjugacy class in GL(2,E'3)i? (the determinant ah ■ T{b)a{a) = aT{b) ■ ba{a) lies in , 
and Es is the fixed field of a in E). We choose Y to be the quadratic extension Es of 
F. The image of this torus is the torus (up to conjugacy) in Cy(F) which splits over the 
biquadratic extension E of F. 

In type IV we have a e E^ , E = E^{^/D), aa'^a e E^ , E3 = F{\^), and the norm 
map becomes 

Here permutes the two diagonal matrices on the right, and permutes each pair of 
eigenvalues. Since both components of N{*) have equal determinants in F^ , diag(*, *) 
defines a conjugacy class in GL(2,i?3)F. Hence the norm map defines a conjugacy class in 
Cy = Cy(F) for each ^-stable conjugacy class of type IV in G = G(F), where we take Y 
to be Es- 

In types I and III the image of the map N does not correspond to any conjugacy class 
in Cy, for any quadratic extension Y of F. 



Jacobians 

The character relation that we study relates the product of the value at t of the twisted 
character of our representation tty = /(3,i)(l3 x xy) by a factor A{t x 6*), with the product 
by a factor Ac{Nt) of the value at Nt of the character of the (trivial) representation Icy 
of Cy which lifts to Try. 

The factors A{t x 9) and AdNt) are defined and computed in [FZ4], pp. 339-340. We 
have 



Ajte) 

Ac{Nt) 



{a-df {b-cf 



ad 



be 



1/2 



Then in case II if t = diag(a, b, rb, aa), a = ai + a2\fD e E^ , b = bi + b2^ AD e E^, 
we get 



A{te) 

Ac{Nt) 



{a-aay- {b - abf 



aaa 



bab 



1/2 



(2a2V^)2 {2b2VADy 



aiD bf - biAD 



1/2 



In case IV, if t = diag(Q;, aa, o""^q;, cr^a), a = a + b\/D, a = ai + a2\/~A, 6 = 61 + 62 V^, 
aa = aa + ab\/aD, a^a — aa — aby/ aD, a — a^a — 2b\/D, a{a — a^a) — 2ab\/aD, and 



A{t9) 
AciNt) 



{a — a'^a)'^ a{a — a'^a) 



aa'^a 



aaa-^a 



1/2 



{AbabfDaD 



(a2 - b'^D)a{a^ - b'^D) 



1/2 
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Characters 

Denote by / (resp. fcy) a complex- valued compactly-supported smooth (thus locally- 
constant since F is nonarchimedean) function on G (resp. Cy)- Fix Haar measures on G 
and on Cy- 

By a G-module n (resp. C^^-module ttcy) we mean an admissible representation ([BZ]) 
of G (resp. Cy) in a complex space. An irreducible G- module n is called 9 -invariant if 
it is equivalent to the G-module ^tt, defined by ^7T{g) — ii{9{g)). In this case there is an 
intertwining operator A on the space of tt with TT{g)A = A'K{6{g j) for all g. Since = 1 we 
have 7r{g)A'^ = A'^7r{g) for all g, and since tt is irreducible A'^ is a scalar by Schur's lemma. 
Wc choose A with A"^ = 1. This determines A up to a sign. When tt has a Whittaker 
model, which happens for all components of cuspidal automorphic representations of the 
adele group GL(4, A), we specify a normalization of A which is compatible with a global 
normalization, as follows, and then put 7T{g x 6) = TT{g) x A. 

Fix a nontrivial character i/» of F in C^, and a character 'ip{u) = ■0(ai^2 + ^2,3 — 03,4) of 
u = (uij) in the upper triangular subgroup U of G. Note that il^{9{u)) — il^iu). Assume that 
TT is a non degenerate G-module, namely it embeds in the space of "Whittaker" functions 
W on G, which satisfy - by definition - W{ugk) = ilj{u)W{g) for all (7 G G, -u G t/, /c in a 
compact open subgroup K\y of as a G-module under right shifts: {T:{g)W){h) — W{hg). 
Then ^tt is non degenerate and can be realized in the space of functions ^W{g) = W{9{g)), 
W in the space of tt. We take A to be the operator on the space of tt which maps W to ^W. 

A G-module n is called unramified if the space of tt contains a nonzero iiT-fixed vector. 
The dimension of the space of JC-fixed vectors is bounded by one if tt is irreducible. If tt is 
^-invariant and unramified, and f 7^ is a iiT-fixed vector in the space of tt, then Avq is a 
multiple of vq (since dK = K), namely Avq = cvq, with c = ±1. Replace A by cA to have 
Avq — Vq, and put 7r(0) = A. 

When TT is (irreducible) unramified and has a Whittaker model, both normalizations of 
the intertwining operator are equal. In this case ip is unramified (trivial on R but not 
on iT~^R, where tt is a generator of the maximal ideal of R), and there exists a unique 
Whittaker function Wq in the space of tt with respect to ip with Wq = 1 on K. It is mapped 
by tt{0) = A to ^Wo, which satisfies ^Wo{k) = 1 for all k in K since K is 6'-invariant. 
Namely A maps the unique normalized (by Wo{K) = 1) JC-fixed vector Wq in the space of 
TT to the unique normalized i^-fixed vector ^Wq in the space of ^tt, and we have ^Wq = Wq. 

For any (admissible) tt and (smooth) / the convolution operator n^fdg) — Jq f{g)TT{g)dg 
has finite rank. If tt is ^-invariant put 7r(/ci^x6') — fa fig)'^{g)'^{^)dg- Denote by tr 7r{fdgx 
9) the trace of the operator 7r{ fdg x$). It depends on the choice of the Haar measure dg, but 
the [twisted) character x-k of tt does not; Xn is a locally-integrable complex-valued function 
on G X ^ (see [C], [H]) which is ^-conjugacy invariant and locally-constant on the ^-regular 
set, with tr 7T{fdg x 6) ^ Jq f{g)xA9 x ^)dg for aU /. 

Local integrability is not used in this work; rather it is recovered for our twisted character. 

Small representation 
To describe the G-module of interest in this paper, take P to be the upper triangular 



TWISTED CHARACTER OF A SMALL REPRESENTATION 



7 



parabolic subgroup of type (3,1), and fix its Levi factor to be M = {m = diag(m3, mi); nis e 
GL(3, F), mi e F^}. It is isomorphic to GL(3, F) x F^. Let 5 denote (as above) the 
character d{p) = | Ad(p)| Lie AT] of P; it is trivial on the unipotent radical N (= F'^) 
of P. Then the value of 5 at p = mn is |m^^detm3|. Denote by /(tti) the G-module 
TT = Ind(5^/^7ri; F, G) normalizedly induced from tti on F to G. It is clear from [BZ] that 
when TTi is self-contragredient and /(tti) is irreducible then /(tti) is ^-invariant, and it is 
unramified if and only if tti is unramified. 

Our aim in this work is to compute the 9-twisted character x-ky GL{4:,F) -module 

mi ) Xy{^i), Xy is a quadratic 
character of , rrii G GL(z,F), hy purely local means. 

We begin by describing a useful model of our representation, in analogy with the models 
of [FK] and [FZ4] of analogous representations /(2,i)(l2) of PGL(3, F) and /(3 1^(13) of 
PGL(4, F). Indeed we shall express tty as an integral operator in a convenient model, and 
integrate the kernel over the diagonal to compute the character of Try. 

Denote by // = //^ the character |U(x) = |x| '^^"•"^-'/^ of F^ , and by xy a quadratic character 
of F^. This pair (/U, Xr) defines a character /xp = IJ's,y,p of F, trivial on A^, by Hp{p) = 

/u((det m3)/mi'^)Xy (mi) ifp = mn and m = ^"^^ j with ms in GL(3, F), mi in GL(1, F). 

If s = 0, then //p = 5^^'^xy-, where viewed as a character on F, xy takes the value 
Xr(mi) at p. Let Ws — be the space of complex- valued smooth functions ip on G 
with ip{pg) = iJ,p{p)'ip{g) for all p in F and in G. The group G acts on Wg by right 
translation: {7rs{g)ip){h) = '^{hg). By definition, 7(3 i)(l3 x xy) is the G-module Wg with 
s = 0. The parameter s is introduced for purposes of analytic continuation. 

We prefer to work in another model Vg = V]^ of the G-module Wg- Let V denote 
the space of column 4-vectors over F. Let Vs be the space of smooth complex-valued 
functions (j) on V — {0} with 0(Av) = //(A)~^xr (A)0(v). The group G acts on Vg by 
(ts((7)0)(v) = /i(det 5f)(/»(*5fv). Let vq 7^ be a vector of V such that the line {Avo;A 
in F} is fixed under the action of *F. Explicitly, we take vq = * (0,0,0,1). It is clear 
that the map Vs Ws, (p ^ ip = ipcj,, where t(;{g) = (rs(5f)0)(vo) = /^{det g)(p{^g\'o), 
is a G-module isomorphism (check that il^Ts{g)<t> — T^si.g)'^ (!>)■, with inverse i/j ^ cj) = (j)^, 
0(v) = iJ,{detg)~^'ip{g) if v = *gvo {G acts transitively on F — {0}). 

For V = ^{x,y,z,t) in V put ||v|| = max(|x|, \y\, \z\, \t\). Let V'^ be the quotient of the 
set of V in y with ||v|| = 1, by the equivalence relation v ~ av if a is a unit in R. 
Denote by FV the projective space of lines in — {0}. If $ is a function on V — {0} with 
$(Av) = |A|~'^$(v) and dv = dx dy dz dt, then #(v)(iv is homogeneous of degree zero. 
Define 

/ ^{v)dv to be / $(v)dv. 

Jfv Jv° 

Clearly we have 

/ $(v)civ = / ^{gv)d{gv) = \detg\ / ^{gv)d\. 
Jfv Jfv Jwv 

Put iy'{x) — \x\ and m = 2{s — 1). Note that = /x_s. Put (w, v) = *wJv. Then 

(^w,^(^)v) = (w,v). 
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1. Proposition. The operator 

Tj:V,^V-s, {Tj(t^){y)^ f (/>(w) |(w, v) Txr ((w, v))(iw, 

J¥V 

converges on Res > 1/2, and satisfies there TjTa{g) — T-s{0{g))Tj for all g in G. 
Proof. We have 

{Tj{Ts{g)m^) = /(T.(^)(/>)(w)|*wJvrxy((w, v))dw 
= //(det£?)/(/)C£?w)|*wJvrxy((w,v))rfw 
= \detg\-'^^{detg)|(|>{w)\'Cg-'w)J^rrxY{Cg-'w,^r))dw 
= Wiv)(det^) /0(w)| W • J-i^-Vvrxr((w, ei'g)v))dw 
= W^v) (det ^) / 0(w) I (w, ^C^)v) ((w, ^C^) v))dw 
= (^^)(det^(^)) . (T,^0)C^(^)v) = [(r_.(^(^)))(Tj0)](v) 

for the functional equation. 

For the convergence, we may assume that = 1 and *v = (0, 0, 0, 1), so that the integral 
is \x\^dx, which converges for Rem > —1. Our m is 2s — 2, as required. □ 

The spaces Vs are isomorphic to the space W of locally-constant complex- valued functions 
(j) on with 0(Av) = xr(A)0(v) for all A e i?^, and is equivalent to an operator T^'^ 
on W. The proof of Proposition 1 implies also 

1. Corollary. The operator T J °^s{g^^) is an integral operator with kernel 
Wz.)(det^(^))|(w,^C^-i)v)rxy((w,^C^-^)v)) (v,w mV^) 

and trace 

tr[Tf'0 o r,(^-i)] = {i^/ij){detg) j \\gJwrxY{'^gJ^)d^. 

Next we normalize the operator T^ = T^'^- Recalling that xy is unramified (= 1 on 
i?^, Xyl'^) = "1)? we normalize T^ so that it acts trivially on the one-dimensional space 
of i^-fixed vectors in Vs. This space is spanned by the function (f)o in Vg with 00 (v) = 1 for 
all V in This is the only case studied in full in this paper. 

Denote again by tt a generator of the maximal ideal of the ring R of integers in our 
local nonarchimcdean field F of odd residual characteristic. Denote by q the number of 
elements of the residue field R/ttR of R. Normalize the absolute value by |7r| = q~^, and 
the measures by vol{|x| < 1} = 1. Then vol{|x| = 1} = 1 — q~^, and the volume of V^ is 
(1 - g-^)/(l - g-i) = 1 + g-i + g-2 + g-3. 



TWISTED CHARACTER OF A SMALL REPRESENTATION 



9 



2. Proposition, //vq = *(0, 0,0, 1), we have 

When s = 0, the constant is {1 + q~'^){l + q)~^ ■ 
Proof. Since xy is unramified, we have 



(T^0o)(vo)= / (^o(v)|VJvorxvCvJvo)dv= / \xrxY{x)dxdydzdt 
Jv° Jv° 

x\^XY{x)dxdydzdt/ 
XY{x)dx/ [ dx = {l + g-2(*+i))/(l + 



vIKi 



v||<l 



dx 



x\=l 



= (1 + g-^-^) 
since m = 2{s — 1) and 

/ \xrxY{x)dx = {l + q-""-^)-^ [ dx. 

J\x\<l J\x\=l 

The proposition follows. 

Character computation for type I 
For the ^-conjugacy class of type I, represented hy g = t ■ diag(r, s, s, r), the product 



□ 



\gJv = {t,z,x,y) 
is equal to 



/air a2-Dr' 

6is h2Ds 

62S his 

\a2V air 



^ 1\ /t^ 

1 I I 2 

-10 X 

V-1 0/ \yj 



—t^a2Dr — z^b2Ds + x'^b2S + y^a2r. 

Note that r and s range over a set of representatives for /Ne/fE^ . 
By Corollary 1, we need to compute 



|rs|^~''|4a262-D| 



{al-alD)ibl-blD)\^/^ 



[ \\gJvrxY{'vgJv)dv 

\af^^~-^\Y{a)dxdydzdt. 



Here a is x'^b2S + y^a2r — z^b2Ds — t^a2-Dr. Put r' = ~ff f • Thus we need to compute the 
value at s = of the product of 

xY{b2s)\ir\4Dr'm^r - - D)r/^ 

s 02 02 02 

with the integral iJir', D), where Q = x^ - ry^ - Dz^ + rDt^ and 

/f(r,D)= / \Q\''^^-^)xY{Q)dxdydzdt. 
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I. Theorem. When Y/F is unramified, the value of ij {y, D) at s = is 0. 

Proof. Consider the case when the quadratic form Q — — ry^ — Dz^ + rDt^ is anisotropic 
(does not represent zero). Thus D = tt and r e — R^'^ (hence |r| = 1, \D\ — 1/g), or 
D E R^ — i?^^ and r = tt. The second case being equivalent to the first, it suffices to deal 

with the first case. 

The domain max{|a;|, \y\, \z\, \t\} = 1 is the disjoint union of {\x\ = 1}, {\x\ < 1, \y\ — 1}, 
< 1, \y\ < 1, \z\ = 1} and {|a;| < 1, \y\ < 1, \z\ < 1, \t\ = 1}. Note that Xy{x'^ - ry^ - 
Dz"^ + rDt^) is equal to 1 on the first two subdomains and equals —1 on the other two. 
Thus the integral /J(r, D) is the quotient by dx of 

dx + / dxdy — / / / dxdydz 

x\=l J J|a;|<l,[y| = l J J J \x\<l,\y\<l,\z\ = l 



-q-"^ / / / / dxdydzdt 

= l + q-^- - = 1 + g-i _ g-2- _ g-2--i. 

The value at s = is and thus the theorem follows when the quadratic form is anisotropic. 

We then turn to the case when the quadratic form is isotropic. Recall that r ranges over 
a set of representatives for /Ne/fE^ , E = F{VD). Thus D e F — F^, and we may- 
assume that \D\ and |r| lie in 

I.l. Proposition ([FZ4]). When the quadratic form x"^ — ry^ — Dz^ + rDt^ is isotropic, 
V lies in Ne/fE^ , and we may assume that the quadratic form takes one of three shapes: 

x'^ -y"^ - Dz'^ + Dt"^, D e R"" - R"""^; + Try^ - ttz^ - tt V ; a;^ - - ttz^ + Trt^ 

The set V*^ — V/ ~, where V = {v — {x,y,z,t) G -R^; max{|x-|, \y\, \z\, \t\} — 1} and ~ is 
the equivalence relation v ~ av for a e i?^, is the disjoint union of the subsets 

V^ = V^{r,D)^Vn{r,D)/^, 

where 

Vn = Vn{r,D) = {v;majc{\x\,\y\,\z\,\t\}=l,\x^ -ry^ - Dz^ +rDt^\ = 

over n > 0, and of {v; x'^ — ry^ — Dz'^ + rDt^ = 0}/ ~, a set of measure zero. 
Thus the integral /J(r, D) coincides with the sum 

oo 

^(-l)>--vol(V;^r,D)). 

n=0 

When the quadratic form represents zero the problem is then to compute the volumes 
vol(F„°(r, D)) = vol(K(r, D))/{1 - 1/g) (n > 0). 
We need some results from [FZ4] : 
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1.0. Lemma ([FZ4]). When (? e i?^^ and n> 1, we have 

1.1. Lemma ([FZ4]). When D =ir and r = 1, thus \rD\ = 1/q, we have 

1-1/q, ifn = 0, 

vo\{V^) - <! q-\l - l/q){2 + 1/q), z/n = 1, 

2g-"(l - 1/q)(1 + l/g), ifn>2. 

1.2. Lemma ([FZ4]). When D — it and r = —tt, thus \rD\ — 1/q^ , we have 



voi(v;°) 



f 1, i/n = 0, 

q-\l-l/q), ifn = l, 

q-^{2-l/q-2/q^), ifn = 2, 

I 2q-^{l-l/q){l + l/q), ifn>S. 



1.3. Lemma ([FZ4]). When E/F is unramified, thus \rD\ = 1, we have 

l-l/g2, ifn^O, 



g-"(l-l/g)(l + 2/g + l/Q^), ifn>l. 



Proof of Theorem I. We are now ready to complete the proof of Theorem I in the isotropic 
case. Recall that we need to compute the value at s = (m = —2) of lY{r,D). Here 
/J(r, D) coincides with the sum 

oo 

J2{-irq-^^yo\iV^ir,D)) 

n=0 

which converges for m > —1 by Proposition 1 or alternatively by Lemmas L1-L3. The value 
at m = — 2 is obtained then by analytic continuation of this sum. 
Case of Lemma I.l. The integral /]^(r, D) is equal to 

oo 

vol(Fo°) - vol(F°) + VOl(F,°) 

n=2 



12 



YUVAL Z. FLICKER AND DMITRII ZINOVIEV 



When m = —2, this is 



Q \ Q r J \ qj \ qj ^ + q 

Case of Lemma 1.2. The integral /J^(r, D) is equal to 

oo 

vol(Vo°) - q-^Yol{Vr)+q-'^yol{V,') + 5^(-l)"g— vol(V„«) 

-2m 









1-1 






q 


K qj 





2fl-ll fl+il g-3(-+l) 



n=3 




1 
















1 + 




qm+ij 



9 9 



When m = —2, this is 

q\ qj q 

Once simplified this is equal to 0. 

Case of Lemma 1.3. The integral /^(r, L>) is equal to 

oo 

voi(Vo°) + 5^(-i)"g-""^voi(v;«) 



qJ 1 + 9 



n=l 



1 

= 1-- 



When m = —2, this is 



q V qJ \ qJ '^ + q 



= 0. 



The theorem follows. □ 

Character computation for type II 
For the 6'-conjugacy class of type II, represented by = t • diag(r, s, s, r), the product 



^vgjyr = {t, z, X, y) 



/air a2Dr\ / 1\ 

6is b2ADs I I 10 

62S 61S 0-100 

\a2r air / \-l 000/ 



z 

X 
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is equal to 

-t^a2Dr - z%2ADs + x^h2S + y^a2V = b2s{x^ - y^r' - z'^AD + t^Dr'). 

Here ai + aav^ G {Ei = F{VD)) and hi + ^av^AD G E^ {E2 = F{y/AD)), and 
r' = — a2r/62S. As r ranges over a set of representatives for F^ /Ne^/fEi (and s for 
F^ /NE2/FE2), we may rename r' by r. 

Thus, by Corollary 1, we need to compute the value at s = of the product of 

XY{h2s)\lr\Av'D^\\{{ff - iffD){{^±f-AD)r/^ 

S O2 O2 O2 

and the value when r is r' and Q = — ry^ — ADz^ + rDt^ of the integral 

lJ{v,A,D)= f \Q\''^'-^^XY{Q)dxdydzdt. 
Jv° 

The property of the numbers A, D and AD that we need is that their square roots gen- 
erate the three distinct quadratic extensions of F. Thus we may assume that {A, D, AD} = 
{w,7r, utt}, where u G — i?^^. Of course with this normalization AD is no longer the 
product of A and D, but its representative in the set {1, w, tt, utt} mod F^^. Since r ranges 
over a set of representatives for F^ /Nej^/fEi , it can be assumed to range over {l,7r} if 
D = u, and over {!,«} if \D\ = \ir\. 

In this section we prove 

II. Theorem. When Y j F is unramified, the value of 

XY{b2s)\4rD^/A\lJ{r, A, i^)/(T^0o)(vo) 

ats = is -2xY{b2s)6{Y, Es). 

RecaU that E3 = F{VA). As usual, 5{Y, E3)isliiY ^E^ and if F ^ E3. 

The meaning of this result is that the twisted character of Try on elements of tori of type 
II relates to values of the trivial character on Cy(F), y = £^3, on the torus which splits 
over E. It docs not relate to such values on Cy (F), Y' ^ E3. 

Recall Lemma II. from [FZ4]. 

II. 0. Lemma ([FZ4]). The quadratic form — ry'^ — AD ^rDf^ takes one of six forms: 
x"^ — y'^ + ■K{t'^ — uz"^), x"^ — uy'^ + uTr{t'^ — z^), x"^ — + ut"^ — wkz"^ , x"^ — y"^ — uz"^ + ■nt'^ , 
— Tty'^ + u7T{t'^ — z'^), — uy'^ — uz"^ + mzt^ , where u G — R^^ . It is always isotropic. 

The set V'^ = V/ ~, where V = {v = (x, y, 2, t) G -R^; max{|a:|, |y|, \t\} = 1} and ~ is 
the equivalence relation v ~ av for o; G i?^, is the disjoint union of the subsets 

= V^{r, A, D) = K(r, A, D)/ ^, 
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where 

Vn = K(r, A, D) = {v; max{|a;|, \y\, \z\, \t\} = 1, \x^ - - ADz'' + rDt^\ = l/g'^}, 

over n > 0, and of {v; — ry"^ — ADz"^ + rDt^ = 0}/ a set of measure zero. 
Since Y/F is unramified, the integral /J^(r, A^ D) is equal to 



5;;(-l)-g— vol(V;°(r,Ai^)). 



n=0 

The problem is then to compute the volumes 

yo\{V^{r,A,D)) = Yol{Vn{r, A, D))/{1 - l/q) (n > 0). 

Choose tt to be a non square unit. To prove Theorem II, by Lemma II. we need precisely 
the following Lemmas from [FZ4]. In Lemmas II. 1 and II. 2, = F{y/A) is unramified 
over F. 

11. 1. Lemma ([FZ4]). When the quadratic form is x'^ — y'^ + 7r(t^ — uz^) {thus r = 1, 
A = u, D = IT up to squares), we have 

I -l/q, ifn = 0, 

vo\{V^) = { 2/q- 1/q^ + l/q\ ifn=l, 
2q-^{l-l/q), ifn>2. 

11. 2. Lemma ([FZ4]). When the quadratic form is — uy^ + ui^it^ — z'^) {thus r = u, 
A = u, D = IT up to squares), we have 

(1 + 1/q, ifn = 0, 

yo\{V^)=l q-^l-l/q), ifn=l, 
[ 2?-("+i)(l - 1/?), ifn>2. 

11. 3. Lemma ([FZ4]). When the quadratic form is x'^—y'^+ut'^—uTTz'^ or x"^ —y'^ —uz'^ +irt'^ , 
we have 

( 1, ifn = 0, 

vol(V;°)=i ifn=l, 
[ ifn>2. 

11. 4. Lemma ([FZ4]). When the quadratic form is x"^ — iry"^ + u'K{t^ — z^), we have 

1, ifn = 0, 

yo\{V^)={ l/q, ifn=l, 
q-^{l-l/q^), ifn>2. 
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II. 5. Lemma ([FZ4]). When the quadratic form is — uy'^ — uz^ + uitt^ , we have 

1, ifn = 0, 

yo\iV^)={ 1/q, i/n=l, 
q-^{l-l/q''), ifn>2. 

Proof of Theorem II. To prove Theorem II, recall that we need to compute the value at 
s = (m = —2) of the product 



XYib2s)\4rDy/A\I^{r, A, i?)/(r%)(vo). 
Here lY{r,A,D) is equal to the sum 

oo 

5^(-l)\-"-vol(V,«(r,i?)) 

n=0 

which converges for m > —1 by Proposition 1 or alternatively by Lemmas II.1-II.3. The 

value at m = — 2 is obtained then by analytic continuation of this sum. 

Case of Lemma II. 1. We have |4rD-\/]4| = 1/q, and the integral /J(r, A, D) is equal to 

oc 

voi(yo°) - g-"^ voi(y°) + voi(v;°) 

When m = —2, this is 



q \q q^ q^ J q^ \ Q J \ q'>^+^ 



q \q q-^ J \ q J i- + q 1 + Q 



Multiplying by |4rD-\/]4| = 1/q we obtain — 2(1 + 1/ q'^)(l+q) ^. We are done by Proposition 
2. 

Case of Lemma II. 2. We have |4rD-\/]4| = 1/q, and the integral /]^(r, A, D) is equal to 

oo 

vol(Fo°) - 9""" vol(F°) + ^(-1)^?-""^ VOl(F„°) 

n=2 



V q) (T q\ qj V 9"^+V 



1 If 1\ 1 2 

1 + . f 1 

q q^ 
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When m = —2, this is 



1 + --1 + - + -1-- -J^_ = 1 + - 

q q q \ qj 1 + q 1 + q \ 



Multiplying by \4rDvA\ = l/qwe obtain 2(1 + l/r)(l + q)~^. 
Case of Lemmas II. 3, II. 4, II. 5. The integral ij {r, A, D) is equal to 

vol(yo°) - q"^ vol(yO) + $](-l)^g-""^ vol(K?) 

n=2 

99" V «y V «y V 9"'+' 



-1 



When m = —2, this is 

1 - , , , - , , - , 



□ 



Character computation for type III 
For the ^-conjugacy class of type III we write out the representative g = t ■ diag(r,r) as 

/airi+a2r2A {air2 + a2ri)A {biri + b2r2A)D {bir2 + b2ri)AD\ 

air2 + a2ri oiri + a2r2A (6ir2 + b2r\)D (6iri + b2r2A)D 

&iri+&2?2^ {bir2 + b2ri)A airi + a2r2A (air2 + a2ri)A 

V bir2 + b2ri biVi + &2^'2^ ai?'2 + ci2ri oiri + a2r2^ / 

The product *\gJv (where *v = (x, j/, 2;, t)) is equal to 

(6ir2 + b2ri){f + z^A - y'^D - x^AD) + 2(&iri + b2r2A){zt - xyD), 

where ai -\-a2\fA G and 61 +62 G i?;^ . The trace is a function of (7, and r = r\ -\-r2\/~A 
ranges over a set of representatives in [E^ — F{\/~A)) for £"3 /Ne/EsE^ . 
Define the quadratic form Q — Q{x, y, z, t) to be 

rb - T(rb) ^ ^2^ _ ^2^ _ ^2^^) ^ ^ T(r6))(^t - xyD). 
2v A 

Set /J(r, A, D) to be equal to 

\Q\^^'~^hY{Q)dxdydzdt. 



I 

Jv° 



The property of the numbers A, D and AD that we need is that their square roots generate 
the three distinct quadratic extensions of F. Thus we may assume that {A, D, AD} = 
{u, TT, utt}, where u G — R^^. Of course with this normalization AD is no longer the 
product of A and D, but its representative in the set {l,u,Tr,uir} mod F^^. 
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III.l. Proposition, (i) If D = u andA = w {or iru) then \fA ^ N e / e^E^ = A^R^ . 
(ii) If A = u and -1 e i^^^ andD = Tr {or iru) then VJ ^ Ne/e^E'' = {-D)^R^^. 
{in) If A = u = -1 ^ i?^^ and D =ir {or iru) then there is d e with + 1 e -R^^ = 
i?^ - R""^, hence d + ieR^ - R^^ {i = y/A) and sod + ie - Ne/e^E'' . 

Proof. For (iii) note that + Tri?} is the multiphcative group of a finite field F of 

q elements. There are 1 + ^(^ — 1) elements in each of the sets {1 + x^; a; G F} and 
{-y^; y G F}. As 2(1 + \{q - 1)) > g, there are x, y with 1 + = -y^. But y 7^ as 
— 1 ^ F^^. Hence there is x with 1 + x'^ ^ F^^, and our d exists. □ 

Since r ranges over a set of representatives for /Ne/e^E^ , by Proposition III.l we 
can choose br to be 1 or \/A 01 d + i. Correspondingly the quadratic form takes one of the 
three shapes 

t^ + z'^A-y^D - x^AD, zt - xyD, or t'^ - z'^ - y'^D + x'^D + 2d{zt - xyD). 

III. Theorem. When Y/F is unramified, the value of 1^ {r, A,D) at s = is 0. 

Proof. Assume that br — \/A ^ Ne/e-^E^ , thus \brT{br)D\ — \AD\., and the quadratic form 
is t^ + z'^A — y'^D — x'^AD. li \ A\ = 1/q or —1 is a square, we can replace A with —A. The 
quadratic form then becomes the same as that of type I. The result of the computation is 
0, see proof of Theorem I, case of anisotropic quadratic forms and we are done in this case. 

lfA = -1, br = d+i ^ Ne/EsE^, the quadratic form is t'^-z'^-y'^D+x'^D+2d{zt-xyD). 
It is equal to - uV^ - D{Z^ - uT'^) with X ^ t + dz, Y = z, Z = y + dx, T ^ x and 
u = d^ + l e R^ — i?^^. Since \D\ = 1/q the quadratic form is anisotropic and the result 
of the computation is by the proof of Theorem I, case of anisotropic quadratic forms. 

Assume that br — 1, thus \brT{br)D\ = \D\ and the quadratic form is zt — xyD. Then it 
is I times {z + 1)"^ — {z — t)^ — D[{x + yY — {x — yY]. Since max{|a;|, \z\, \t\} = 1 implies 
max{|a; + y|, \x — \z + t|, \z — t\} = 1, the result of the computation is by the proof of 
Theorem I, cases of Lemmas I.l and 1.3. The theorem follows. □ 

Character computation for type IV 

For the 6'-conjugacy class of type IV we write the representative g = t • diag(r,r) (where 
t = h~^t*h, t* = diag(Q;, era, a^a, a'^a)) as 

/ain + a2r2A (air2 + a2ri)^ {b[ri + 6^r2^)-D {b[r2 + b'2ri)AD\ 

a\r2 + a2ri airi + a2r2A {b'ir2 + b'2ri)D {b'^ri + b'2r2A)D 

biri+b2r2A {bir2 + b2ri)A airi + a2r2A {air2+a2ri)A 

\ 6ir2 + b2ri biri + b2r2A air2 + a2ri airi + a2r2A / 

Here E^ = F{y/A) is a quadratic extension of F and E = Es{y/D) is a quadratic extension 
of E3, thus Ae F - F^ and D = di+ d2VA e E3 - E|, d^ G F. 

If —1 G -F^^ we can and do take D — \/A, where A is a nonsquare unit u if E^/E is 
unramified, or a uniformizer ir if E^/F is ramified. If —1 ^ F^'^ and E^/F is ramified, once 
again we may and do take A = Tr and D = y/A. 
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If —1 ^ and E^^/F is unramified, take A = —1 and note that a primitive 4th root 
C = ■i of 1 hes in (and generates it over F). Then E/E^ is unramified, generated by 
\/D, D = di + id2, and we can (and do) take d2 — 1 and a unit di = d in F^ such that 
d'^ + 1 ^ F^"^. Then D = d + i ^ E^^. The existence of d is shown as in the proof of 
Proposition III.l. 

Further a = a + h\fD e E^ , where a = ai + a2\fA e E^ , b = bi + b2\/A e E^ , and 
r = n + r2A/A e E^/Ne/e^E''. The relation = + b2y/A defines 6i = + 62^2^ 
and 62 = 62*^1 + &i<^2- 

RecaU from CoroUary 1 that we need to compute 

(^) /, l'v^'^v|-x.('v,Jv)dv. (.) 

Since detgr = ar ■ a{ar) ■ a^{ar) ■ a'^{ar), we have 



{ar — a'^ (ar))'^ a{ar — a'^{ar))'^ ^^"^ 



ara'^{ar) a{ar)a^{ar) 
\4brDa{brD)\ 



|r2(a2 -62£))(j(r2(a2 -62£)))|V2• 
When s = 0, this is \brDa{brD)\. 

The product WgJv (where *v = (x, y, 2, t)) is then equal to 

{bir2 + b2ri){t^ + z^A) - {b[r2 + 62^1) (y^ + x^A) + 2{biri + b2r2A)zt - 2{b[ri + b'2r2A)xy. 

Since bD = b[ + 62 AM, this is 

^!__^(t2 + ^2^) ^ ^ 

6rL> - (TibrDl^^^ ^ ^^^^ _ ^ a{hrD))xy. 



lyfA 

Note that r ranges over a set of representatives for E^ I^EjEz^^ 1 ^^id b lies in E^ . As 
b is fixed, we may take br to range over E^ / N ^ / e.,E^ ■ 

Further, note that E^/F is unramified if and only if E/E^ is unramified. Hence br can 
be taken to range over {l,'7r} if E^/F is unramified, and over {l,w} if E^/F is ramified, 
where tt is a uniformizer in F and u is a nonsquare unit in F, in these two cases. Thus in 
both cases we have that a{br) = br, and the quadratic form is equal to brQ, where 

Q = Q{x, y, z, t) = 2zt - ^~J'^\ y^ + x'^A) - {D + a{D))xy. 

2v A 

Thus we need to compute the value at s = of the product of \brDa{brD)\, xy(6r)|6r|2(*~i) 
and 



lJ{v,A,D)^ f \Q\^(^-'^XY{Q)dxdydzdt. 
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IV. Theorem. When Y j F is unramified, the the value of 

XY{br)\br\'^'-'^brDa{brD)\lY{r,A,D)/{T^cj>o)M 

at s = is -2xY{br)6{Y,E3). 

To prove this theorem we need some results from [FZ4]. 
IV. 1. Proposition ([FZ4]). Up to a change of coordinates, the quadratic form 

2zt - ^ {y' + x'A) -{D + a{D))xy 

is equal to either +7ry'^ — 2zt or x'^ — uy"^ — 2zt with u G — R^^ . It is always isotropic. 
Recall that Y/F is unramified. Then the integral I^{r,A,D) is equal to 

oo 

^(-irg-"-voi(v;°(r,Ai^)). 

n=0 

The problem is then to compute the volumes 

vol(F,°(r, A, D)) = vol(V;,(r, A, D))/{1 - 1/g) (n > 0). 
IV. 2. Lemma ([FZ4]). When the quadratic form is x^ — uy^ — 2zt, we have 

l + l/g2, ^fn = 0, 



g-'*(l - l/g)(l + 1/g^), if n > 1 



Proof of Theorem IV. To prove Theorem IV, recall that we need to compute the value at 
s = (m = —2) of /J(r, A, D). Since Y/F is unramified, the integral /J(r, ^, D) coincides 
with the sum 



vol(F,°(^,i;)) 



n=0 

which converges for m > — 1. The value at m = —2 is obtained then by analytic continuation 
of this sum. 

Case of + ity'^ — 2zt. Make a change of variables z ^ 2u~^z', followed by z' i-^ z. Thus 
the quadratic form is equal to 

-u-^((z-tf - (z + tf -ux'^ -uiry^). 

Note that up to a multiple by a unit, this is a form of Lemma II. 3. Since maxjl^l, = 1 
implies max{|2; + t\,\z — t\} — 1, the result of that lemma holds for our quadratic form as 
well. In this case E^/F is ramified, and our integral is zero. 
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Case of — uy"^ — 2zt. This is the case where E^/F is unramified. By Lemma IV. 2, the 
integral 



/r (r, A D) = voKFo") + $^(-1)^9-^"^ vol(K?) 



is equal to 



When m — —2, this is 



-1 



1 + ^ + ( 1 - - ) ( 1 + ^ ) = ( 1 + 4 

1 \ Q/\ / q + l 1 + q \ q^ 

The theorem follows by Proposition 2. □ 
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